Within a mixed-reality environment, a user can interactive with synthetic actors and synthetic objects (hereafter collectively referred to as “synthetic objects”). The term “synthetic object” is intended to include, but is not limited to, any object that can be rendered on a display. The term “mixed-reality environment” is intended to include, but is not limited to, a visual domain wherein a real world user can interact with synthetic objects integrated into a user's view of the real world. For example, when implemented as a training tool for soldiers, a mixed-reality environment may allow for a soldier to interact with synthetic objects that represent enemy troops, wherein the synthetic troops will appear to exist in the real world. According to another example, in an implementation of a mixed-reality environment utilized for gaming (e.g., a tennis match), a user may interact with a synthetic object (e.g., an opponent) and one or more synthetic objects (e.g., a tennis balls).
Currently, virtual reality systems are the primary means of providing a user the ability to interact with synthetic objects. Virtual reality systems create a completely synthetic environment within which a user can interact with synthetic objects. Given that conventional virtual reality systems facilitate user interaction with synthetic objects within an entirely synthetic environment, virtual reality systems do not render synthetic objects in a real life visual domain which allows a user to view a synthetic object within a real world context. As such, virtual reality systems fail to address certain technical obstacles confronted when rendering synthetic objects that appear within the user's perception of the real world. For example, given that virtual reality systems do not capture real world video data, these systems fail to address low latency processing of such real world video data when calculating a user's pose. As such, creation of a mixed-reality environment presents technical issues that are not addressed when creating a virtual reality environment.
Conventional systems and methods for creating a mixed-reality environment also fail to address many of the obstacles confronted when rendering a realistic mixed-reality user experience. For instance, conventional systems and methods for creating a mixed-reality environment fail to utilize low latency video processing in determining a user's pose. Without the use of low latency processing, synthetic objects rendered in a mixed-reality environment may appear to jitter or bounce within a user's field of vision. Such movement may detract for a user's ability to properly interact with the mixed-reality environment. In addition, existing mixed-reality applications fail to accurately calculate the pose of a user-controlled device.
To effectively integrate the actions of a user-controlled device into a mixed-reality environment, the pose of the user-controlled device must be accurately estimated. Accurate estimation of a user-controlled device pose is needed to generate a realistic interpretation of the motion or action of a user-controlled device. For example, in a mixed-reality environment wherein a real world user is simulating a tennis match against a synthetic opponent, the pose of the user's tennis racket must be accurately calculated to determine if the user makes contact with the synthetic tennis ball, in order to translate the effect of returning the synthetic opponent's serve into the mixed-reality environment. Conventional methods and systems for creating mixed-reality environments lack efficient processes, systems, and devices for calculating the pose of a user-controlled device.
The prior art lacks a mixed-reality environment that can effectively meet the current needs within the training and gaming sectors. More specifically, the prior art lacks a method and system for providing low latency processing of a user's pose together with the accurate calculation of the relative pose of a user-controlled device.